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ABSTRACT 


The  acquisition  of  new  weapon  systems  is  a  long  and  expensive  process.  Test 
and  Evaluation  is  a  critical  and  costly  step  in  that  process.  Model-Test-Model 
(MTM)  is  an  emerging  operations  research  technique  that  can  improve  efficiency  and 
effectiveness  of  weapon  system  evaluation.  For  example,  MTM  can  be  used  to 
predict  whether  a  given  test  scenario  will  yield  significant  results. 

The  goal  of  this  thesis  is  to 

(1)  examine  the  suitability  of  proposed  scenarios  for  comparing  the  existing 
antiarmor  system,  Dragon  versus  the  proposed  system.  Javelin; 

(2)  compare  the  weapon  systems  according  to  approved  measures  of 
effectiveness  (MOE)  and, 

(3)  identify  potential  additional  test  conditions  which  may  cause  significant 
changes  in  the  MOE. 

The  pre-test  model  using  Janus(A),  a  high  resolution,  combat  simulation  will 
impact  on  the  operational  test  of  the  weapon  systems  occurring  in  September  1993. 
Although  M-T-M  is  a  DOD  approved  construct,  limited  experiments  have  been  done. 

This  thesis  is  unique  because  it  explores  for  the  first  time,  using  MTM,  an 
infantry  weapon  system  with  approved  scenarios  and  MOE;  secondly,  this  thesis  has 
been  delivered  in  time  to  impact  on  the  Operational  Test  and  Evaluation  plan.  The 
results  will  directly  benefit  Operational  Test  and  Evaluation  Command  (OPTEC)  in 
their  test  design  and  operational  effectiveness  analysis. 
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I.  INTRODUCTION 


A.  THESIS  OBJECTIVE 

The  acquisition  of  new  weapon  systems  is  a  long  and  expensive  process.  Test 
and  Evaluation  is  a  critical  and  costly  step  in  that  process.  [Ref.  1,  Ref.  2] 
Model-Test-Model  is  a  relatively  new  concept  being  promoted  as  an  operations 
research  technique  by  which  DoD  can  conduct  more  efficient  and  effective  test  and 
evaluation  of  proposed  weapon  systems  as  well  as  improve  the  validity  of  existing 
modeling  tools  or  identify  needed  improvements.  [Ref.  3,  Ref.  4] 

Model-Test-Model  (MTM)  consists  of  three  phases.  The  first  phase  examines 
the  proposed  experimental  design  of  the  test  in  order  to  identify  possible  efficiencies 
in  and  effectiveness  improvements  of  that  design.  [Ref.  3]  To  date  only  limited 
pre-test  modeling  of  proposed  or  new  weapon  systems  have  been  conducted  prior  to 
actual  testing.  This  limited  pre-test  modeling  included  an  Air  Defense  weapon 
system  and  an  Armor  weapon  system,  but  never  to  the  level  of  fidelity  of  Infantry 
weapon  systems.  [Ref.  5,  Ref.  6]  Additionally,  even  though  some  pre-test  modeling 
has  been  done,  none  of  the  pre-test  modeling  results  have  been  conducted  in  time 
to  impact  on  the  actual  experimental  design.  As  such,  a  chasm  exists  between  the 
theoretical  impact  of  MTM  and  empirical  proof  that  existing  modeling  tools  have  the 
capability  to  accomplish  these  aims. 
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This  thesis  explores  the  suitability  of  pre-test  MTM  concepts  for  an  infantry 
weapon  system.  Secondly  this  thesis  investigates  the  possible  impact  pre-test 
modeling  would  have  on  the  experimental  design  of  the  operational  test  of  U.S. 
Army’s  current  Medium  Antitank  Weapon  system,  Dragon  versus  a  proposed 
replacement  -  Javelin.  The  test  is  scheduled  in  1993  at  Fort  Hunter- Liggett, 
California.  In  order  to  address  these  objectives,  this  research  applies  MTM  concepts 
using  Test  and  Experimentation  Command  (TEXCOM)  developed  scenarios. 
Janus(A),  the  Army’s  principal  combined  arms,  interactive,  high  resolution  model  is 
the  tool  used  to  generate  research  data.  [Ref.  7]  The  data  for  the  two  weapon 
systems  are  evaluated  according  to  three  critical  Measures  of  Effectiveness  (MOE) 
which  are  based  on  Critical  Operational  Issues  and  Criteria  (COIC)  of  the  test.  [Ref. 
8] 


B.  ISSUES 

In  regards  to  the  suitability  of  MTM  concepts  the  specific  issue  addressed  in 
this  research  is  whether  or  not  the  representations  of  Infantry  actions  in  Janus(A)  are 
a  realistic  portrayal  of  corresponding  actual  Infantry  actions.  This  issue  will  be 
addressed  primarily  through  a  literature  search  and  discussion  of  applicable  Infantry 
tactics  and  their  corresponding  representation  in  Janus(A).  In  regards  to  the 
efficiency  and  effectiveness  of  the  experimental  design,  sjjecific  issues  to  be  addressed 
include; 

(1)  whether  or  not  the  proposed  scenarios  are  plausible  and  facilitate 
examination  of  the  differences  between  the  competing  weapon  systems,  and 
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(2)  identification  additional  issues  which  could  impact  on  the 

efficiency/effectiveness  of  the  test  design  and  employment  of  the  weapon 
system. 


C.  BACKGROUND  ON  INFANTRY  ANTIARMOR  SYSTEMS 

From  the  late  1960’sthe  Army  developed  the  Light  Antitank  Weapwn  (LAW), 
Medium  Antitank  Weapon  (MAW)  and  Heavy  Antitank  Weapon  (HAW)  doctrine 
for  infantry  forces  to  combat  enemy  armored  vehicles  and  weapon  systems.  The 
LAW  is  a  short-range,  light-  weigh;,  weapon  that  a  soldier  carries  in  addition  to  his 
primary  or  assigned  weapon.  The  LAW  evolved  from  the  M-72,  a  66  milli-meter 
rocket  with  an  approximate  200  meter  range  to  the  current  AT-4  which  has  a  larger 
84  millimeter  high  explosive  projectile  and  a  greater  300  meter  range.  [Ref.  9] 
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Figure  1  Law  Ounner  in  Action 
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The  HAW  is  a  heavy  or  crew-served  weapon  system.  Its  nomenclature  is  the 
TOW  II  (Tube-Launched,  Optically-Tracked,  Wire  Command  Link  Guided)  missile. 
Although  the  TOW  II  can  be  ground-mounted,  it  is  usually  vehicle  mounted.  In 
mechanized  units  the  TOW  II  is  mounted  on  the  Bradley  Fighting  Vehicle  and  the 
Improved  TOW  Vehicle.  It  is  mounted  on  the  High  Mobility  Multipurpose  Wheeled 
Vehicle  (HMMWV)  in  light  infantry  units  and  Cobra  Attack  Helicopters  in  aviation 
units.  It  has  a  range  of  3750  meters  and  a  thermal  night  sight.  [Ref.  9]  The  TOW 
II  warhead  has  been  modified  to  defeat  reactive  armor. 


Figur*  2  TOW  Mountad  on  HMMWV 
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The  current  MAW  is  the  Dragon.  It  is  man-carried  and  is  the  primary 
antiarmor  weapon  system  at  platoon  and  company  level.  The  Dragon’s  guidance 
system  is  similar  to  the  TOW  in  that  it  is  optically  sighted,  command-link 
wire-guided.  It  has  a  1000  meter  range  and  a  thermal  night  sight.  [Ref.  9] 

Presently  the  Dragon  gunner  is  required  to  guide  a  relatively  slow-moving 
missile  to  the  target.  The  range  of  most  enemy  tank  main  gun  and  heavy  machine 
gun  is  1500  -  1800  meters.  Therefore,  the  Dragon  gunner  remains  exposed  for  up 
to  1 1  seconds  within  the  enemy’s  engagement  range.  This  assumes  the  Dragon 
gunner  fires  at  maximum  engagement  range.  Even  using  proper  flank  and  rear  shot 
tactics,  the  gunner  is  still  at  risk.  Also,  improved  armor  has  degraded  the  weapon’s 
lethality.  Given  the  deficiencies  of  limited  range,  gunner  vulnerabilities,  and  eroded 
lethality,  changes  in  tactics  have  not  been  able  to  sufficiently  address  the  mission 
need.  [Ref.  11] 
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Javelin  is  under  development  to  address  the  cited  deficiencies.  The  Advanced 
Antitank  Weapon  System  -  Medium  (AAWS-M)  or  the  Javelin  is  designed  to  have 
an  extended  range  of  up  to  2200  meters.  Javelin  is  designated  as  a  fire  and  forget 
weapon  system  which  permits  a  gunner  to  acquire  a  target,  fire  on  the  target  and 
immediately  resume  a  covered  and  concealed  position.  The  Javelin’s  guidance 
system  is  based  on  an  infrared  imagery  seeker.  The  Javelin’s  missile  when  launched 
travels  more  than  two  times  faster  than  the  Dragon.  Because  the  Javelin  has  a  top- 
attack  mode  as  well  as  a  direct  line  of  sight  capability  its  inherent  lethality  is 
increased.  Theoretically,  a  top-attack  mode  permits  the  missile  to  penetrate  a 
thinner  more  vulnerable  top  than  the  better  protected  sides  and  front  glacias. 


Figure  4  JAVELIN 
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n.  MODEI^TEST-MODEL  CONCEPT 


A.  MODEL-TEST-MODEL  PHASES 

Although  Department  of  Defense  policy  for  applying  modeling  and  simulation 
in  support  of  Operational  Test  and  Evaluation  has  been  in  existence  since  1989, 
MTM  as  presently  applied  is  a  relatively  new  concept.  The  MTM  concept  applies 
modeling  directly  in  support  of  the  operational  test  and  evaluation  process.  This 
concept  envisions  the  synergistic  interaction  of  testing  and  modeling  in  order  to 
positively  impact  both  operational  testing  and  the  combined  arms  modeling  process. 

Mr.  Hollis,  Deputy  Under  Secretary  of  Army  for  Operations  Research, 
approved  and  directed  that  additional  MTM  concepts  be  incorporated  into  Army 
Test  and  Evaluation  Policy  in  1991.  [Ref.  3,  Ref.  4]  Since  then  DOD  has  also 
approved  MTM.  [Ref.  14,  Ref.  15]  MTM  has  three  distinct  phases.  The  first  phase 
is  pre-test  modeling  which  simulates  proposed  test  scenarios.  The  objective  of 
pre-test  modeling  is  to  develop  an  efficient  and  effective  experimental  design. 
Pre-test  modeling  iteratively  refines  and  adjusts  the  test  design  in  order  to  balance 
test  objectives  with  test  costs. 

The  second  phase  is  the  test  phase.  While  testing  is  conducted,  modelers 
observe  trials  and  data  collection  methods.  Particular  attention  is  given  to  definition 
of  each  data  collection  stq)  in  order  to  insure  test  data  corresponds  to  model  data. 
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The  post  test  modeling  phase  focuses  on  examining  the  degree  of  association 
between  model  and  test  results.  Normally  this  examination  entails  fixing  selective 
parameters  while  statistically  evaluating  the  association  of  repetitive  modeling  trials 
with  corresponding  test  trials.  This  phase  can  also  be  used  to  extend  test  results 
beyond  the  tested  scenarios  and  conditions.  The  rationale  behind  extending  test 
results  is  to  gain  insight  into  the  utility  of  the  proposed  weapon  system  where  active 
testing  costs  may  be  prohibitive  or  testing  constrained  by  safety  or  environmental 
consideration.  One  caution,  however,  is  the  further  the  extension  or  extrapolation 
is  from  the  calibration  points  the  less  credible  the  model  results  may  become. 

The  M1A2  tank  and  the  Line  of  Sight  Forward  Heavy  (LOSF-H),  an  air 
defense  artillery  system,  have  been  partially  evaluated  under  the  MTM  Concept. 
Both  the  LOSF-H  and  M1A2  were  considered  in  a  post-test  evaluation.  Pre-test 
modeling  on  the  M1A2  was  done  with  notional  scenarios  and  MOE’s.  [Ref.  S,  Ref. 
6] 

This  MTM  thesis  research  on  the  Javelin  is  unique  for  many  reasons.  It 
represents  the  first  infantry  system  in  the  test  and  evaluation  process  to  be  explored 
using  MTM.  Secondly,  this  research  represents  the  first  time  actual  scenarios,  terrain 
and  measures  of  effectiveness  will  be  used  in  the  pre-test  modeling  of  the  operational 
test.  Finally,  this  thesis  represents  the  first  time  pre-test  modeling  will  be  conducted 
in  time  to  impact  on  the  actual  test  design. 
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B.  DESCRIPTION  OF  JANUS(A) 


The  simulation  used  to  conduct  the  pretest  model  is  Janus(A).  Janus  is  named 
for  the  Roman  god  who  was  the  guardian  of  portals  and  the  patron  of  beginnings  and 
endings.  Janus(A)  is  an  interactive,  two-sided,  closed,  stochastic,  combined  arms 
simulation.  Interactive  refers  to  decision  making  during  the  simulation  or 
man-in-the-loop.  Two-sided  refers  to  two  opposing  forces.  Closed  means  that 
opposing  force  distributions  information  is  limited.  Stochastic  refers  to  the 
problematic  way  results  such  as  direct  fire  engagements  are  determined.  Combined 
arms  means  the  principal  focus  is  on  combined  arms  forces  and  maneuver.  Janus(A) 
also  models  weather,  visibility,  chemical  environment,  mines  and  any  other  combat 
variables.  Terrain  is  depicted  with  contour  lines,  vegetation,  roads,  waters  and  urban 
areas.  Theoretically  terrain  is  represented  by  cells  which  correspond  to  Defense 
Mapping  Agency  elevation,  vegetation,  and  cultural  feature  description.  Graphic 
symbols  represent  one  or  more  systems  and  each  system  has  one  or  more  wes^ns. 
For  example  the  symbol  for  an  infantryman  may  represent  a  nine  man  squad  where 
each  infantryman  carries  an  M-16  rifle  and  a  light  antitank  weapon.  Combat 
between  systems  in  the  Janus  simulation  is  initiated  based  on  a  standard  Army 
detection  algorithm  for  weapon  system  sensor  capabilities.  In  addition  to  limitations 
imposed  by  sensor  system  capability,  detections  occur  only  if  physical  line  of  sight 
exists  between  the  sensor  and  the  target.  If  a  system  has  line  of  sight,  can  range  the 
target,  has  ammunition,  and  is  not  in  hold-fire  status,  then  the  system  fires  the  most 
appropriate  weapon  system  at  the  target.  Appendix  A  shows  a  photograph  of  a 
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Janus(A)  display.  The  simulation  resolves  engagements  by  comparing  random 
number  draws  to  a  probability  of  hit  and  kill  database.  Postprocessing  files  allow 
extensive  data  collection  on  the  detection  and  engagement  process.  [Ref.  12] 
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in.  SCENARIOS/MEASURES  OF  EFFECTIVENESS 


Essential  to  any  research  on  the  suitability  of  MTM  concepts  for  Infantry 
Weapon  Systems  is  development  of  realistic  scenarios  which  employ  those  weapons. 
TEXCOM  -  developed  scenarios  for  the  Initial  Operational  Test  (lOT)  of  the  Javelin 
provide  the  basis  of  the  scenarios  evaluated  in  this  research.  The  scenarios  examined 
in  the  model  and  the  scenarios  scheduled  for  conduct  during  the  test  correspond. 
The  scenarios  vary  tactically  and  fall  into  offensive  and  defensive  operations. 
Modeled  scenarios  include  hasty  defense,  deliberate  defense,  deliberate  attack  and 
movement  to  contact/hasty  attack  (MTC).  Conditions  in  these  scenarios  are 
systematically  varied.  Conditions  include  day  operations  with  and  without  Mission 
Oriented  Protective  Posture  (MOPP)  equipment.  Because  Javelin  has  a  long  wave 
Infrared  Sight,  differences  in  the  sensor  between  day  and  night  conditions  in  the 
model  are  minimal.  Further,  night  parameter  data  on  Dragon/Javelin  was  not 
available  for  modeling  given  the  time  constraints  of  this  thesis. 

MOPP  conditions  vary  for  each  scenario.  Hence  MOPP  levels  are  factored  into 
the  Janus(A)  modeling  process.  MOPP  refers  to  the  protective  overgarments  to 
include  boots,  gloves  and  mask  that  soldiers  wear  in  a  Nuclear,  Biological  or 
Chemical  (NBC)  contaminated  area.  A  higher  MOPP  level  reduces  soldier 
effectiveness  and  increases  reaction  time.  In  MOPP4  soldiers  wear  protective 
clothing  as  well  as  a  mask  for  a  simulated  NBC  environment.  For  this  research,  the 
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MOPP  level  for  the  hasty  defense  in  this  analysis  is  MOPP4.  The  other  three 
scenarios  were  modeled  in  MOPPO.  The  deliberate  defense,  deliberate  attack  and 
MTC  scenarios  were  modeled  under  daylight  conditions. 

Modeled  force  sizes  correspond  with  projected  force  sizes  for  the  operational 
test.  For  the  scenarios  modeled  this  was  platoon  size  blue  forces  and  company  size 
red  forces  for  the  hasty  and  deliberate  defenses.  For  the  deliberate  attack  and  MTC, 
company  size  blue  forces  and  platoon  size  red  forces  were  modeled.  It  is  also 
important  to  note  the  blue  platoons  and  companies  modeled  are  light  or  dismounted 
infantry.  A  blue  platoon  consists  of  30  soldiers  organized  in  three  squads.  Each 
platoon  possesses  two  Dragon/Javelin  Antitank  weapons  (MAW),  two  M-60  machine 
guns,  six  Squad  Assault  Weapons  or  light  machine  guns,  12  M-203  grenade 
launchers,  and  eight  M-16  rifles.  Each  rifleman  also  carries  2  AT-4’s.  Each  blue 
company  is  made  up  of  three  platoons.  The  blue  force  sizes  and  Dragon/Javelin 
complements  are  displayed  in  Table  1. 


TABLE 1 


BLUE  FORCE  S 

IZE 

HASTY 

DEF 

MOPP4 

DELIBERATE 

DEFENSE 

DELIBERATE 

ATTACK 

MOVEMENT 

TO  CONTACT 

BLUE 

SOLDIERS 

30 

30 

90 

MAW 

SYSTEMS 

2 

2 

6 

6 

12 


The  red  force  company  possess  three  T-72  tanks  and  eight  BMP  mechanized 
infantry  fighting  vehicles  for  a  total  of  1 1  vehicles.  Red  force  platoons  possess  two 
T72’s  and  two  BMP’s.  The  red  force  sizes  are  displayed  in  Table  2. 


TABLE 2 


1  RED  FORCE  SI 

ZE 

1 

HASTY 

DEF 

MOPP4 

DEUBERATE 

DEFENSE 

DEUBERATE 

ATTACK 

MOVEMENT  | 
TO  CONTACT 

RED 

VEHICLES 

11 

11 

4 

4 

The  MOE’s  considered  are  derived  from  the  TEXCOM  Critical  Operational 
Issues  and  Criteria  (COIC)  selected  for  the  Initial  Operation  Test  which  will  occur 
in  September  1993  at  Fort  Hunter-Liggett,  California. 

They  are  as  follows; 

-  Engagement  Range 

-  average  engagement  range  in  meters 

-  Lethality 

-  It  kills/tf  shots  fired 

-  Survivability 

-  tt  fiiendly  soldiers  surviving/ 
if  friendly  soldiers  starting 

The  MOE’s  above  were  selected  for  a  number  of  reasons.  First,  they  are 
directly  addressed  in  COIC.  They  are  quantifiable  and  easily  derived  from  Janus(A) 
post-processing  files.  Other  MOE’s  included  in  the  COIC  but  not  included  in  this 
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research  have  less  to  do  with  the  weapon  system  and  more  to  do  with  the  gunner. 
Examples  include  performance  by  soldiers  with  selected  fitness  scores  or  suitability 
criteria  concerning  manpower  and  personal  integration  (MANPRINT).  These 
issues/MOE’s  cannot  be  described  adequately  in  Janus  at  this  time  and  if  included 
would  cloud  weapon  versus  weapon  comparison. 

Another  condition  examined  within  the  context  of  this  thesis  is  basic  load.  This 
is  not  a  condition  that  is  addressed  in  the  TEXCOM  MOE  or  measures  of 
performance,  but  may  impact  on  COIC’s  such  as  Lethality  and  Survivability.  Basic 
load  is  the  number  of  rounds  available  per  weapon  system  for  a  given  scenario.  Basic 
load  varies  according  to  mission  as  well  as  preparation  time.  For  an  offensive 
operation  a  Dragon/Javelin  gunner  will  carry  the  weapon  sight/launch  unit  and  one 
round  and  the  assistant  gunner  will  carry  a  spare  round.  In  the  defensive  operations, 
preparation  time  for  a  hasty  defense  is  less  than  24  hours.  The  unit  may  be  limited 
to  the  rounds  available  in  the  platoon  or  what  is  in  the  company  trains  which  may 
range  from  three  to  five.  In  the  deliberate  defense  with  24-72  hours  to  prepare  the 
company  has  time  to  resupply  rounds  from  the  battalion  trains  and  up  the  basic  load 
to  six.  [Ref.  16]  See  Table  3, 
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TABLES 


BASIC  LOAD 

HASTY  DEF 
MOPP4/ 

INC  BL 

DELIBERATE 

DEFENSE 

DELIBERATE 

ATTACK 

MOVEMENT 

TO 

CONTACT 

BASIC  LOAD 

3/5 

6 

2 

2 

Preparation  time  impacts  on  other  differences  in  hasty  and  deliberate  defenses. 
Soldiers  in  a  deliberate  defense  may  have  more  or  better  obstacle  plans  as  well  as 
better  prepared  fighting  positions  which  offer  greater  survivability.  The  differences 
between  a  deliberate  attack  and  a  MTC  have  to  do  with  amount  of  intelligence 
known  about  the  enemy's  location  and  disposition.  In  a  deliberate  attack  a  unit 
knows  the  enemy’s  disposition  and  location  and  therefore  can  plan  and  execute 
protected  routes  to  the  enemy’s  location.  In  a  MTC  a  unit  is  trying  to  find  the 
enemy  to  make  contact  and  destroy  him.  The  chosen  scenarios  and  conditions  offer 
a  broad  cross-section  of  missions  for  the  system  to  be  tested.  Imbedding  the  system 
into  unit  level  operations  will  produce  more  relevant  data  for  analysis. 


rv.  DISCUSSION  OF  SUITABILITY  OF  JANUS(A)  REPRESENTATION 


Janus(A)  is  similar  to  operational  testing  for  direct  fire  engagements.  Janus(A) 
uses  line-of-sight  algorithms  while  tests  use  lasers  attached  to  the  weapon.  In  both 
simulation  and  reality  the  firing  system  must  have  an  unobstructed  line-of-sight  to  the 
target  to  register  a  hit.  In  Janus(A),  Infantry  soldiers  are  represented  as  standing 
upright  when  they  are  moving.  In  reality  soldiers  crawl  or  move  in  crouched 
positions  as  well  as  upright.  The  inability  of  Janus(A)  to  model  various  individual 
soldier  movements  is  overshadowed  by  the  level  of  resolution  in  its  terrain  modeling. 
At  present  the  finest  degree  for  terrain  resolution  is  12.5  meter  grids  or  squares.  In 
Janus(A),  soldiers  in  defensive  positions  can  be  in  full  or  partial  defilade.  If  a  soldier 
is  in  full  defilade  he  and  his  weapon  cannot  detect  targets  or  fire  at  them.  Real 
fighting  positions  fully  prepared  provide  cover,  concealment  and  overhead  protection. 
Therefore  a  soldier  with  an  antitank  weapon  could  actually  fire  from  a  full  defilade 
position.  Janus(A)  can  be  run  interactively  or  non-interactively.  For  this 
research  Janus(A)  scenarios  were  allowed  to  run  non-interactively.  This  means 
systems  did  not  deviate  from  their  original  preplanned  route  of  advance  or  direction 
of  attack.  A  more  realistic  approach  might  be  a  man-in  —the-  loop  so  that  the  battle 
can  be  fought  by  reactive  or  thinking  opponents.  The  reason  the  simulated  battles 
were  not  fought  interactively  was  to  vary  as  few  parameters  as  possible.  Therefore 
the  data  generated  for  the  MOE’s  was  primarily  weapon  dependent  and  not  man-in- 
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the-loop  dependent.  Secondly,  the  stopping  criteria  for  either  side  was  reaching  their 
march  objective.  A  designated  level  of  casualties  sustained  is  another  stopping 
criteria  that  may  have  more  reasonably  modeled  reality.  The  march  objective 
stopping  criteria  was  selected  because  it  was  consistent  and  could  be  repeated  for 
analytical  purposes.  It  also  generated  a  larger  data  population  which  gives  more 
robustness  to  the  analysis.  Kills  given  hits  in  Janus(A)  are  binary  in  nature.  Once 
a  hit  is  generated  it  is  selected  as  a  casualty  or  suppression.  Categorical  kills  for 
vehicles  such  as  catastrophic,  mobility  and  weapon  system  may  represent  reality 
more  closely,  but  are  not  available  in  Janus(A).  As  previously  mentioned  Janus(A) 
does  not  allow  systems  to  rire  from  a  full  defilade  position.  The  system  can  be 
changed  interactively  from  full  to  partial  defilade  so  the  system  can  fire.  This 
artificiality  may  cause  a  slight  degradation  in  survivability  in  the  model  not  found  in 
the  test.  These  modeling  issues  need  to  be  addressed  to  close  the  gap  between 
simulation  and  reality  but  the  important  issue  of  survivablility  is  one  of  detection  and 
accurate  probability  of  hit/probability  of  kill  (ph/pk).  It  is  important  to  note  that  the 
actual  ph/pk’s  for  all  weapon  systems  used  as  input  are  not  included  to  keep  the 
thesis  unclassified.  The  detection  algorithm  in  Janus(A)  has  been  validated  in  other 
studies.  The  ph/pk’s  are  generated  by  analysis  done  at  TRADOC  Analysis 
Command  at  Monterey  and  will  be  refined  with  input  from  other  Army  agencies  and 
as  further  testing  is  completed. 
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V.  EFTICIENCY  AND  EFFECTIVENESS  OF  THE  EXPERIMENTAL  DESIGN 

A.  METHODOLOGY 

In  order  to  examine  whether  or  not  the  proposed  f^narios  are  plausible  and 
facilitate  examining  the  differences  between  the  competing  weapon  systems,  five 
Janus(A)  runs  were  conducted  for  each  weapon  system  for  all  scenarios  under  the 
varied  conditions.  Although  only  five  runs  were  made,  the  analysis  represents  from 
30-60  total  data  points  for  each  MOE.  Means  of  these  data  points  represent  each 
of  the  five  runs. 

The  basic  analytical  objective  is  to  determine  if  the  proposed  scenarios  revealed 
a  statistical  difference  between  the  two  weapon  systems  given  the  same  scenarios. 

A  number  of  analytical  tools  were  used  to  evaluate  data  resulting  from  this 
research.  The  first  analytical  tool  used  is  notched  box  and  whisper  plots.  Box  plots 
provide  useful  information  about  location,  dispersion  and  skewness.  A  notch  is 
added  to  each  box  corresponding  to  a  confidence  interval  around  the  median,  while 
the  width  of  the  box  is  proportional  to  the  square  root  of  the  number  of  observations 
in  the  data  set.  [Ref.  18,  Ref.  19]  Pair  wise  comparisons  are  made  between  the 
weapon  systems  by  examining  whether  the  notches  overlap.  The  next  tool  used  is 
One  Way  Analysis  of  Variance  (ANOVA).  This  technique  assumes  normality,  a 
common  variance  and  independent  samples.  The  hypothesis  testing  for  each  MOE 
varies  according  to  scenario. 
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In  each  case  the  hypothesis  tested  were: 

Hq  :  M|  =  M2 

H.  :  M,  Mj 

Where: 

M,  =  mean  of  Dragon  for  each  MOE 
Mj  =  mean  of  Javelin  for  each  MOE 

Next  Bartletts  test  is  used  to  test  for  common  variance  between  the  Dragon  and 
Javelin.  See  Appendix  B. 

Finally  the  Kruskal- Wallis  One  Way  Analysis  by  Ranks  is  used.  This 
nonparametric  technique  is  chosen  because  it  makes  no  assumptions  concerning 
shape  and  location  parameters  and  thus  provides  robustness  to  analytical  analysis 
performed. 

B.  RESULTS 

1.  Hasty  Defense  in  MOPP4. 

-  Average  Engagement  Range:  Analysis  of  notched  box  plots  for  the 
hasty  defense  in  MOPP4  indicates  the  range  for  the  Javelin  is  more  than  double  the 
Dragon’s  range.  The  Javelin’s  median  for  average  engagement  range  is 
approximately  1970  meters  while  the  Dragon’s  median  is  approximately  960  meters. 
Little  deviation  exists  from  the  median  for  both  weapon  systems.  This  is  reasonable 
because  both  weapon  systems  have  optimal  fields  of  fire  and  will  fire  close  to  their 
maximum  engagement  ranges.  The  ANOVA  and  KW  values  support  differences  in 
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the  weapon  systems  to  a  five  percent  level  of  significance.  See  Figure  5  and  Table 
4. 
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-  Lethality:  Lethality  remains  under  35%  for  the  Dragon  while  the 
Javelin  ranges  from  80—  100%  with  the  median  near  100%.  What  appears  to  be  an 
outlier  for  the  Dragon  is  a  valid  data  point  r^resenting  six  engagements.  The 
lethality  of  49%  is  slightly  higher  than  the  median  but  within  the  Dragon’s  ph/pk. 
The  ANOVA  and  KW  tests  support  rejection  of  the  null  hypothesis  of  equal 
means/means  of  rank.  See  Figure  6  and  Table  4. 


Figur*  C 
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-  Survivability:  There  is  no  significant  difference  between  the  weapon 
systems  in  survivability.  The  Dragon’s  median  is  approximately  93%  while  the 
Javelin’s  median  is  around  90%  with  an  extremely  wide  confidence  interval.  ANOVA 
and  KW  results  show  no  differences  in  the  weapon  systems.  See  Figure  7  and  Table 
4. 
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TABLE  4  ANOVA/KW  VALUES  FOR  HASTY  DEFENSE  IN  MOPP4 


ANOVA 

_  KW  1 

Test 

Statistic 

P-Value 

Test 

Statistic 

P-Value 

Range 

46076.1 

.0000 

6.859 

.0088 

Lethality 

115.193 

.0000 

7.500 

.0062 

Survivability 

.334 

.5852 

.1895 

.6633 

2.  Deliberate  Defense. 


-  Average  Engagement  Range:  As  occurred  for  Hasty  Defense  in 
MOPP4  the  medians  of  average  engagement  range  are  statistically  different  between 
the  weapon  systems  in  deliberate  defense.  The  Dragon’s  median  is  950  meters  and 
the  Javelin’s  is  just  under  2000  meters.  ANOVA  and  KW  analysis  support  differences 
between  the  weapons.  See  Figure  8  and  Table  5. 
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-  Lethality;  Median  for  the  Dragon’s  lethality  is  approximately  50% 
while  the  Javelin  is  over  95%.  ANOVA  and  KW  results  support  differences  between 
the  Dragon  and  Javelin  to  a  five  percent  level  of  significance.  For  this  scenario  there 
appears  to  be  an  outlier  for  the  Javelin  below  the  median.  Due  to  the  randomness 
of  the  simulation  fewer  first  round  hits  were  scored  on  that  run  but  it  remains  a  valid 
data  point.  See  Figure  9  and  Table  5. 
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-  Survivability:  Median  for  the  Dragon  is  around  93%  while  the  median 
for  the  Javelin  is  100%.  The  confidence  intervals  do  not  overlap  thus  showing 
differences  in  the  medians  for  the  weapons.  ANOVA  and  KW  values  support  the 
alternative  hypothesis  that  the  means/means  of  rank  are  not  equal.  The  survivability 
is  a  constant  100%  for  the  Javelin  because  the  blue  force  possessed  enough  rounds 
to  destroy  all  enemy  vehicles  before  they  were  able  to  kill  any  blue  soldiers.  See 
Figure  10  and  Table  5. 


Figure  10 
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TABLES  ANOVA/KW  VALUES  FOR  DELIBERATE  DEFENSE 


ANOVA 

KW  1 

Test 

P  - 

Test 

P  - 

Statistic 

Value 

Statistic 

Value 

14514.2 

.0000 

6.818 

.0090 

42.885 

.0002 

7.031 

.0080 

23.929 

.0012 

7.867 

.0050 

3.  Deliberate  Attack. 


-  Average  Engagement  Range:  Analysis  of  the  deliberate  attack 

scenarios  indicate  that  the  medians  are  different.  Median  for  the  Dragon  is  750 
meters  and  the  Javelin  is  just  over  1700  meters.  ANOVA  and  One  Way  Analysis  of 
Ranks  show  distinct  means/means  of  rank  for  the  weapons.  See  Figure  1 1  and  Table 
6. 


Figur*  11 
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-  Lethality:  Medians  for  lethality  are  dissimilar.  The  Dragon’s  median 
is  under  25%  while  the  Javelin’s  is  50%.  Again  their  confidence  intervals  do  not 
overlap.  Their  ANOVA  and  KW  values  display  differences  to  a  five  percent  level 


of  significance.  See  Figure  12  and  Table  6. 
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-  Survivability:  The  medians  for  survivability  are  distinct.  The  Dragon’s 
is  81%  and  the  Javelin’s  is  97%.  ANOVA  and  KW  analysis  support  differences  to  a 


five  percent  level  of  significance.  See  Figure  13  and  Table  6. 


Figure  13 


TABLE  6  ANOVA/KW  VALUES  FOR  DELIBERATE  ATTACK 


ANOVA 

KW  1 

Test 

Statistic 

P  - 
Value 

Test 

Statistic 

P  - 
Value 

Range 

525.526 

.0000 

6.818 

.0090 

Lethality 

7.374 

.0264 

4.930 

.0263 

Survivability 

6.989 

.0295 

4.645 

.0311 
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4.  Movement  to  Contact. 


-  Average  Engagement  Range:  Analysis  of  the  Movement  to  Contact 
scenarios  show  significant  differences  for  the  medians.  The  Dragon’s  median  is 
approximately  960  meters  while  the  Javelin’s  is  close  to  1720  meters.  ANOVA  and 
KW  analysis  support  dissimilar  means/means  of  rank  for  the  weapon  systems.  See 
Figure  14  and  Table  7. 
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-  Lethality:  Medians  for  lethality  are  statistically  unique.  The  Dragon’s 
lethality  is  under  25%  while  Javelin’s  is  65%.  ANOVA  and  KW  analysis  reject  the 


equality  of  means  and  means  of  rank.  See  Figure  15  and  Table  7. 


Figur*  IS 
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-  Survivability;  The  survivability  for  the  two  weapon  systems  are  again 


statistically  different.  The  Dragon’s  median  is  under  60%  while  the  Javelin’s  is 
approximately  98%.  Values  for  ANOVA  and  KW  reject  similarities  to  a  five  percent 
level  of  significance.  See  Figure  16  and  Table  7. 


Figure  16 


TABLE?  ANOVA/KW  VALUES  FOR  MOVEMENT  TO  CONTACT 


ANOVA 

_ KW  _ J 

Test 

Statistic 

P  - 
Value 

Test 

Statistic 

P  -  1 

Value 

Range 

331.679 

.0000 

6.818 

.0090 

Lethality 

9.863 

.0138 

4.961 

.0259 

Survivability 

36.295 

.0003 

6.859 

.0088 
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5.  Hasty  Defense  in  MOPP4  with  Increased  Basic  Load 

-  Average  Engagement  Range:  Results  of  the  Hasty  Defense  in  MOPP4 
with  Increased  Basic  Load  display  dissimilarities  in  the  medians  in  the  weapon 
systems  for  average  engagement  range.  The  Dragon’s  median  is  approximately  960 
meters  and  the  Javelin’s  median  is  1980  meters.  ANOVA  and  KW  analysis  support 
differences  between  the  means  and  means  of  rank  to  a  five  percent  level  of 
significance.  See  Figure  17  and  Table  8. 


Ti . ; . : . i 


Figure  17 
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-  Lethality:  Differences  in  the  medians  are  again  apparent.  The 
Dragon’s  median  is  30%  and  the  Javelin’s  is  80%.  ANOVA  and  KW  also  show 
statistical  differences  to  a  five  percent  level  of  significance.  See  Figure  18  and  Table 
8. 
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-  Survivability;  There  are  statistical  differences  between  the  medians  of 
the  weapons.  The  Dragon’s  median  survivability  is  90%  while  the  Javelin’s  is 
approximately  93%.  From  ANOVA  and  KW  analysis,  their  means/ means  of  rank 
are  dissimilar  to  a  ten  percent  level  of  significance.  See  Figure  19  and  Table  8. 


Figure  19 


TABLES  ANOVA/KW  VALUES  FOR  HASTY  DEFENSE  SCENARIOS 


ANOVA 

KW 

Test 

Statistic 

P  -  Value 

Test 

Statistic 

P  -  Value 

Range 

354766 

.0000 

6.944 

.0084 

Lethality 

6250 

.0000 

9.000 

.0026 

Survivability 

4.234 

.0736 

4.033 

.0446 

6.  Trends. 


Given  the  terrain  modeled  in  Janus,  analysis  supports  the  assumptions  that 
the  current  operational  tests  planned  for  the  MAW  will  support  determining 
differences  between  the  competing  weapons  systems.  Pretest  modeling  indicates  that 
across  all  scenarios  and  conditions  average  engagement  range  for  the  Javelin  remains 
over  two  times  that  of  the  Dragon.  For  defensive  operations  the  Dragon’s  lethality 
varies  from  35%-50%  while  Javelin’s  lethality  varies  from  80%-l(X)%.  Lethality  is 
lower  in  offensive  operations.  It  varies  from  22%-28%  for  the  Dragon  and  50%-65% 
for  the  Javelin.  The  lower  lethality  for  offensive  operations  is  intuitive  because  in 
defensive  operations  the  weapons’  fields  of  fire  are  selected  to  maximize  the  tactical 
advantage  and  therefore  maximize  lethality.  In  offensive  operations  the  gunners 
employ  their  weapon  system  at  the  point  they  detect  the  enemy  and  can  engage  the 
target.  That  location  may  not  necessarily  have  optimal  fields  of  fire.  Survivability 
shows  trends  not  only  in  offensive  versus  defensive  operations  but  also  shows 
sensitivity  to  basic  load.  Javelin  displays  a  greater  survivability  in  the  offensive 
operations  and  as  the  basic  load  is  varied  in  the  defensive  operations. 

It  is  important  to  understand  what  the  MOE  ratios  represent  for  lethality 
and  survivability.  In  the  defensive  operations  the  difference  in  survivability  equates 
to  one  to  three  soldiers  lives  per  platoon  or  expanding  to  battalion  level  nine  to 
twenty-seven  more  soldiers  survive.  The  number  of  lives  saved  is  even  more 
staggering  for  the  offensive  operations.  Fourteen  to  thirty  more  soldiers  survive  per 


35 


platoon.  That  represents  almost  a  company  at  battalion  level  or  one  quarter  of  the 
fighting  force  in  a  five  company  battalion. 

As  stated  earlier,  conservation  of  testing  resources  is  as  important  today 
as  it  has  ever  been.  The  pretest  modeling  results  indicate  close  similarities  between 
the  defensive  operations  -  hasty  defense  with  increased  basic  load  and  deliberate 
defense,  as  well  as  the  offensive  operations  -  deliberate  attack  and  movement  to 
contact.  Using  one  defensive  scenario  and  one  offensive  scenario  is  a  consideration 
which  would  reduce  and  streamline  testing  resources  to  include  saving  potentially 
millions  of  dollars.  [Ref.  20]  This  postulation  only  considers  issues  addressed  in  the 
discussed  MOE  and  not  those  of  suitability. 
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VI.  CONCLUSION/RECOMMENDATIONS 


The  first  issue  examined  in  this  thesis  is  whether  or  not  Janus(A)  could 
adequately  represent  the  differences  between  the  MAW  systems.  In  other  words,  is 
the  model  suitable  for  MAW  MTM  purposes.  Such  physics  based  issues  as  lethality, 
range  and  speed  of  the  round  as  well  as  velocity  of  dismounted  soldiers  and  vehicles 
are  represented  in  Janus(A)  adequately.  Those  attributes  are  simply  data  inputs. 
Due  to  the  lack  of  object  representation  in  Janus,  replication  of  reality  in  terms  of 
terrain  and  vegetation  is  not  achieved  in  Janus.  However,  if  the  notion  of 
probabilistic  representations  of  terrain  and  vegetation  is  acceptable,  then  the  Janus 
representation  is  acceptable.  In  that  case,  whether  or  not  a  soldier  is  exposed  to  fire 
or  not  can  be  expressed  in  the  probabiilty  of  his  detection.  Of  a  smaller  impact, 
the  difference  between  being  able  to  acquire  and  fire  from  a  full  defilade  position 
is  an  issue  that  needs  to  be  addressed  if  the  model  is  to  more  nearly  approach  reality. 

The  second  issue  of  whether  the  proposed  scenarios  are  sufficient  or  effective 
to  examine  the  differences  between  the  weapon  systems  is  affirmed.  Modeling  the 
TEXCOM  weapon  scenarios  indicate  that  the  differences  in  the  weapon  systems, 
shown  by  the  MOE,  should  result  during  the  operational  test.  Using  various  analysis 
tools  to  investigate  the  Medians,  Means  and  Means  of  Rank  all  three  support  the 
alternate  hypothesis  that  the  median/means/means  of  ranks  of  the  MOE  are 
statistically  different  for  the  weapon  systems.  Engagement  range,  lethality  and 
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survivability  for  the  deliberate  defense,  deliberate  attack  and  movement  to 
contact/hasty  attack  are  different  to  a  five  percent  level  of  significance. 

Basic  load  is  a  condition  that  surfaces  as  an  identified  issue  which  impacts 
directly  on  one  of  the  primary  MOE  -  survivability.  Survivability  becomes  significant 
at  a  five  percent  level  when  the  basic  load  was  increased  from  three  to  five  rounds 
per  system  for  the  hasty  defense  in  a  Chemical/NBC  environment.  Hence,  basic  load 
is  an  important  issue  identified  by  this  analysis  and  not  addressed  in  the  COIC  but 
should  be  prior  to  the  operational  test.  Based  on  the  results  of  the  initial  pre-test 
modeling  in  Janus(A)  the  given  scenarios  and  MOE  are  acceptable  to  show 
differences  in  the  weapon  systems  if  the  systems  are  employed  on  the  same  or  similar 
terrain  at  Fort  Hunter  -  Liggett,  which  maximizes  the  capabilities  of  the  weapon 
systems.  Finally,  this  thesis  considered  reduction  of  the  scheduled  testing  for  possible 
test  savings.  The  pre-test  modeling  results  show  enough  similarities  between  the  two 
defensive  scenarios  and  offensive  scenarios  so  that  the  operational  test  could  be 
streamlined  to  make  it  more  efficient. 
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APPENDK  A 


Appendix  A  is  a  color  copy  of  the  Janus(A)  screen  display.  It  represents  terrain 
from  Fort  Hunter  Liggett,  California  and  is  a  platoon  deliberate  defense.  The  enemy 
is  attacking  from  West  to  East.  The  red  line  represents  a  Javelin’s  fire  and  kill  of 
a  T-72  tank. 
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APPENDK  B 


Bartletts  Test 

Test  Statistic 

P  -  Value  1 

Hasty  Defense  MOPP4 

1 

Range 

1.25 

0.20 

Lethality 

1.02 

0.70 

Survivability 

1.86 

0.04 

Hasty  Defense  M0PP4/Increased  Basic  Load 

1 

Range 

1.01 

0.79  1 

Lethality 

0 

^  1 

Survivability 

1.05 

0.56  1 

Deliberate  Defense 

1 

Range 

2.24 

0.02 

Lethality 

1.00 

0.99 

Survivability 

0 

0 

Deliberate  Attack 

1 

Range 

1.07 

0.47 

Lethality 

1.00 

0.87 

Survivability 

8.33 

0.00 

Movement  to  Contact 

1 

Range 

1.50 

0.09 

Lethality 

1.03 

0.63 

Survivability 

1.50 

0.09 
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